Skip to content

JS-2774 Consume the shared identifier inventory. - #8325

Draft
ZiadMarey wants to merge 1 commit into
masterfrom
zm/JS-2774
Draft

ZiadMarey wants to merge 1 commit into
masterfrom
zm/JS-2774

Conversation

@ZiadMarey

@ZiadMarey ZiadMarey commented Nov 4, 2026 •

Copy link
Copy Markdown

Summary by Gitar

  • Shared inventory:
    • Added identifier-urls.ts loading identifier-urls.json from Analyzer Commons and exporting isKnownIdentifierUrl for S5332
  • Tests:
    • Added identifier-urls.test.ts checking the inventory is empty and that no URL matches

This will update automatically on new commits.

@hashicorp-vault-sonar-prod

hashicorp-vault-sonar-prod Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

JS-2774

@gitar-bot

gitar-bot Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
CI failed: All five CI error templates trace back to one root cause: TypeScript cannot resolve the new `@sonarsource/analyzer-commons-configurations/identifier-urls.json` import in `identifier-urls.ts`, which breaks the unit-test, build, bridge, and Knip/bbf jobs. The PR itself introduces that import, so the failures are change-related.

Overview

Five unique error templates from 5 logs (commit 4281d21) all show the same TS2307 error at packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:18. The PR's new import of @sonarsource/analyzer-commons-configurations/identifier-urls.json cannot be resolved, and the failure cascades into the Maven bridge build and skipped downstream modules. No unrelated or flaky failures were identified.

Failures

Unresolved identifier-urls.json import (TS2307) (confidence: high)

  • Type: dependency
  • Affected jobs: 126509674301 (Unit tests JavaScript), 126509674697 (Build SonarJS on Linux), 126509674648 (Knip), plus the Bridge build jobs (Linux and Windows) that failed in the same way
  • Related to change: yes
  • Root cause: packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:18 imports @sonarsource/analyzer-commons-configurations/identifier-urls.json, and TypeScript reports TS2307 (cannot find module or type declarations). The logs do not establish whether the package is absent, the JSON subpath is not exported, or TypeScript lacks JSON module configuration. The unit-test job stopped at metadata generation before tests ran, and the test file importing the same module would also be affected.
  • Suggested fix: Ensure the required @sonarsource/analyzer-commons-configurations version is declared and installed, and that its package exports include identifier-urls.json at this subpath. If the file exists but TypeScript cannot type it, add JSON module resolution or a declaration. Otherwise switch to the package's supported import path.

Bridge build cascade (Maven exec failure) (confidence: high)

  • Type: build
  • Affected jobs: Bridge build in the Linux and Windows Maven runs
  • Related to change: yes
  • Root cause: The exec-maven-plugin 'Build the bridge' goal exits with code 2 because of the same TS2307 error. Later modules (CSS, Plugin, Standalone, IT) are skipped as a consequence.
  • Suggested fix: Resolving the TS2307 import above should unblock this build step. No separate fix is needed.

Summary

  • Change-related failures: 5 of 5 templates (TS2307 on identifier-urls.json, with the Bridge build cascade and the Knip/bbf and unit-test failures all tracing to it).
  • Infrastructure/flaky failures: None identified.
  • Recommended action: Fix the @sonarsource/analyzer-commons-configurations dependency or export for identifier-urls.json, or correct the import path and JSON typing in identifier-urls.ts, then re-run CI.
Code Review 🚫 Blocked 0 closed / 4 findings

🟡 Medium risk · The new S5332 URL parser and shared-inventory matcher may misclassify identifier URLs at edge cases if consumed by the rule.

Adds identifier-urls.ts to load the shared identifier inventory for S5332 and export isKnownIdentifierUrl, with tests. Blocked: the pinned @sonarsource/analyzer-commons-configurations package does not publish identifier-urls.json, so the import fails and the build and new test cannot run. Also open: the tests do not exercise the matching logic, isKnownIdentifierUrl is not yet called by the S5332 rule, and padded URLs match in the Java filter but not here.

🚨 Cross-Repo: Commons npm package doesn't publish identifier-urls.json

📄 packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:18 📄 packages/analysis/src/jsts/rules/S5332/identifier-urls.test.ts:19

Fix here and in SonarSource/sonar-analyzer-commons

Breaks in: SonarSource/sonar-analyzer-commons - sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json - files

This file imports @sonarsource/analyzer-commons-configurations/identifier-urls.json. In SonarSource/sonar-analyzer-commons, the files list in sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json ships only README.md, secret-patterns.json and secret-exclusion-corpus.json. The 2.31.0-5284 tarball that SonarJS pins matches that: I downloaded it and it has no identifier-urls.json. So the module can't be resolved, and TypeScript build, bundling and the new test all fail. Commons has to generate and publish the inventory in a new npm release, and this repo has to move its pin to that release.

Here: Once Commons publishes a release that contains identifier-urls.json, bump @sonarsource/analyzer-commons-configurations in package.json and regenerate package-lock.json. Until then, keep this PR in draft so master doesn't break.

There: In sonar-analyzer-commons-configurations, generate identifier-urls.json (the domain-to-regex map) into the npm packaging dir, add it to files in src/main/packaging/npm/package.json, and publish a new version.

In SonarSource/sonar-analyzer-commons - sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json:

"files": [
  "README.md",
  "secret-patterns.json",
  "secret-exclusion-corpus.json",
  "identifier-urls.json"
]
💡 Quality: Tests never exercise the URL parsing or matching logic

📄 packages/analysis/src/jsts/rules/S5332/identifier-urls.test.ts:23-29 📄 packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:31-41

Both tests depend on the inventory being empty. One asserts identifierUrls deep-equals {}, which will fail as soon as Commons ships any entry. The other only checks that an empty map returns false. So the authority/remainder regex, lowercasing only the authority, keeping the port, adding / before ?/# suffixes, and running the per-domain regex all go untested. A bug in any of them would pass. Drive isKnownIdentifierUrl with a fixture inventory (for example, factor the matcher to accept a Map<string, RegExp>) and test port, case, query, fragment and non-matching paths. Drop the assertion that the upstream data is empty.

💡 Quality: isKnownIdentifierUrl is never called by the S5332 rule

📄 packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:31

The PR title says S5332 now consumes the shared identifier inventory. But isKnownIdentifierUrl is only imported by its own test: rule.ts, rule.lib.ts and rule.aws.ts don't reference it. So the rule still reports http:// identifier URLs exactly as before and the change has no effect on behavior. Either call the matcher from the cleartext-protocol check in rule.lib.ts (with the AST-usage check the comment requires), or describe the PR as adding the helper only.

💡 Cross-PR: Padded URLs match in the Java filter but not in SonarJS · SonarSource/sonar-analyzer-commons #590

📄 packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:20 📄 packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:34

Breaks when sonar-analyzer-commons#590 is released with a populated inventory and the S5332 rule checks a URL string that has leading or trailing whitespace.

How

  1. CleartextProtocolFilter.isKnownIdentifier calls url.strip() before matching (commons/src/main/java/org/sonarsource/analyzer/commons/appsec/CleartextProtocolFilter.java:244).
  2. isKnownIdentifierUrl runs /^http:\/\/.../i directly on value with no trim, so any surrounding whitespace makes exec return null (packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:34).
  3. So the same padded identifier URL is excluded by Java analyzers but still reported by SonarJS, even though the comment at identifier-urls.ts:20 says the matching semantics are kept aligned.

Fix

  • This PR: Trim value the same way as Java's strip() before running the authority regex, and add a test for a padded URL.
  • sonar-analyzer-commons#590: Nothing needed. Optionally list the whitespace stripping in the README's consumer matching rules.

Compared 4281d21 with sonar-analyzer-commons#590 at c0c18f9

🤖 Prompt for agents
Code Review: Adds `identifier-urls.ts` to load the shared identifier inventory for S5332 and export `isKnownIdentifierUrl`, with tests. Blocked: the pinned `@sonarsource/analyzer-commons-configurations` package does not publish `identifier-urls.json`, so the import fails and the build and new test cannot run. Also open: the tests do not exercise the matching logic, `isKnownIdentifierUrl` is not yet called by the S5332 rule, and padded URLs match in the Java filter but not here.

1. 💡 Quality: Tests never exercise the URL parsing or matching logic
   Files: packages/analysis/src/jsts/rules/S5332/identifier-urls.test.ts:23-29, packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:31-41

   Both tests depend on the inventory being empty. One asserts `identifierUrls` deep-equals `{}`, which will fail as soon as Commons ships any entry. The other only checks that an empty map returns false. So the authority/remainder regex, lowercasing only the authority, keeping the port, adding `/` before `?`/`#` suffixes, and running the per-domain regex all go untested. A bug in any of them would pass. Drive `isKnownIdentifierUrl` with a fixture inventory (for example, factor the matcher to accept a `Map<string, RegExp>`) and test port, case, query, fragment and non-matching paths. Drop the assertion that the upstream data is empty.

2. 💡 Quality: isKnownIdentifierUrl is never called by the S5332 rule
   Files: packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:31

   The PR title says S5332 now consumes the shared identifier inventory. But `isKnownIdentifierUrl` is only imported by its own test: `rule.ts`, `rule.lib.ts` and `rule.aws.ts` don't reference it. So the rule still reports `http://` identifier URLs exactly as before and the change has no effect on behavior. Either call the matcher from the cleartext-protocol check in `rule.lib.ts` (with the AST-usage check the comment requires), or describe the PR as adding the helper only.

3. 🚨 Cross-Repo: Commons npm package doesn't publish identifier-urls.json
   Files: packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:18, packages/analysis/src/jsts/rules/S5332/identifier-urls.test.ts:19

   **Fix here and in [`SonarSource/sonar-analyzer-commons`](https://github.com/SonarSource/sonar-analyzer-commons)**
   
   **Breaks in:** [`SonarSource/sonar-analyzer-commons`](https://github.com/SonarSource/sonar-analyzer-commons) - [`sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json`](https://github.com/SonarSource/sonar-analyzer-commons/blob/HEAD/sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json) - `files`
   
   This file imports `@sonarsource/analyzer-commons-configurations/identifier-urls.json`. In SonarSource/sonar-analyzer-commons, the `files` list in `sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json` ships only `README.md`, `secret-patterns.json` and `secret-exclusion-corpus.json`. The `2.31.0-5284` tarball that SonarJS pins matches that: I downloaded it and it has no `identifier-urls.json`. So the module can't be resolved, and TypeScript build, bundling and the new test all fail. Commons has to generate and publish the inventory in a new npm release, and this repo has to move its pin to that release.
   
   **Here:** Once Commons publishes a release that contains identifier-urls.json, bump `@sonarsource/analyzer-commons-configurations` in package.json and regenerate package-lock.json. Until then, keep this PR in draft so master doesn't break.
   
   **There:** In sonar-analyzer-commons-configurations, generate identifier-urls.json (the domain-to-regex map) into the npm packaging dir, add it to `files` in src/main/packaging/npm/package.json, and publish a new version.
   
   In SonarSource/sonar-analyzer-commons - `sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json`:
   
   ````
   "files": [
     "README.md",
     "secret-patterns.json",
     "secret-exclusion-corpus.json",
     "identifier-urls.json"
   ]
   ````

4. 💡 Cross-PR: Padded URLs match in the Java filter but not in SonarJS · [SonarSource/sonar-analyzer-commons #590](<https://github.com/SonarSource/sonar-analyzer-commons/pull/590>)
   Files: packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:20, packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:34

   **Breaks when** [sonar-analyzer-commons#590](<https://github.com/SonarSource/sonar-analyzer-commons/pull/590>) is released with a populated inventory and the S5332 rule checks a URL string that has leading or trailing whitespace.
   
   **How**
   
   1. `CleartextProtocolFilter.isKnownIdentifier` calls `url.strip()` before matching (`commons/src/main/java/org/sonarsource/analyzer/commons/appsec/CleartextProtocolFilter.java:244`).
   2. `isKnownIdentifierUrl` runs `/^http:\/\/.../i` directly on `value` with no trim, so any surrounding whitespace makes `exec` return null (`packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:34`).
   3. So the same padded identifier URL is excluded by Java analyzers but still reported by SonarJS, even though the comment at `identifier-urls.ts:20` says the matching semantics are kept aligned.
   
   **Fix**
   
   - **This PR:** Trim `value` the same way as Java's `strip()` before running the authority regex, and add a test for a padded URL.
   - **[sonar-analyzer-commons#590](<https://github.com/SonarSource/sonar-analyzer-commons/pull/590>):** Nothing needed. Optionally list the whitespace stripping in the README's consumer matching rules.
   
   _Compared `4281d21` with [sonar-analyzer-commons#590](<https://github.com/SonarSource/sonar-analyzer-commons/pull/590>) at `c0c18f9`_

Verification Rules 1/1 met Repos 1 inspected · PRs 1 inspected

Functional validation No results

Rules 1 passed 2 not applicable

✅ Repository license compliance

Broader impact Repos 1 inspected PRs 1 inspected
Cross-repo 1 repository inspected 1 open impact

SonarSource/sonar-analyzer-commons Inspected

Cross-PR 1 PR inspected 1 open impact

PR 590: ACOMMONS-237 Export the S5332 identifier URL inventory through the shared npm package Inspected

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.
Unblock → Override a blocking verdict and allow merging.

Comment with these commands to change the behavior for this request:

Auto-apply Compact Unblock
gitar auto-apply:on         
gitar display:verbose         
gitar unblock         

Was this helpful? React with 👍 / 👎 | Gitar

@datadog-sonarsource

datadog-sonarsource Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Pipelines

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 5 Pipeline jobs failed

Build | Build SonarJS on Linux — 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

Build | Build SonarJS on Windows — 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

Build | Knip — 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

View all 5 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4281d21 | Docs | View more details | Give us feedback!

Comment on lines +23 to +29
it('uses an empty identifier inventory', () => {
assert.deepEqual(identifierUrls, {});
});

it('does not match URLs when the inventory is empty', () => {
assert.equal(isKnownIdentifierUrl('http://identifier.invalid/schema'), false);
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: Tests never exercise the URL parsing or matching logic

Both tests depend on the inventory being empty. One asserts identifierUrls deep-equals {}, which will fail as soon as Commons ships any entry. The other only checks that an empty map returns false. So the authority/remainder regex, lowercasing only the authority, keeping the port, adding / before ?/# suffixes, and running the per-domain regex all go untested. A bug in any of them would pass. Drive isKnownIdentifierUrl with a fixture inventory (for example, factor the matcher to accept a Map<string, RegExp>) and test port, case, query, fragment and non-matching paths. Drop the assertion that the upstream data is empty.

Was this helpful? React with 👍 / 👎

* CleartextProtocolFilter.isKnownIdentifier(), is not exposed through that package.
* SonarJS therefore performs URL parsing and regex matching here in Node.js using the shared data.
*/
export function isKnownIdentifierUrl(value: string): boolean {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: isKnownIdentifierUrl is never called by the S5332 rule

The PR title says S5332 now consumes the shared identifier inventory. But isKnownIdentifierUrl is only imported by its own test: rule.ts, rule.lib.ts and rule.aws.ts don't reference it. So the rule still reports http:// identifier URLs exactly as before and the change has no effect on behavior. Either call the matcher from the cleartext-protocol check in rule.lib.ts (with the AST-usage check the comment requires), or describe the PR as adding the helper only.

Was this helpful? React with 👍 / 👎

* along with this program; if not, see https://sonarsource.com/license/ssal/
*/

import identifierUrls from '@sonarsource/analyzer-commons-configurations/identifier-urls.json' with { type: 'json' };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Cross-Repo: Commons npm package doesn't publish identifier-urls.json

Fix here and in SonarSource/sonar-analyzer-commons

Breaks in: SonarSource/sonar-analyzer-commons - sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json - files

This file imports @sonarsource/analyzer-commons-configurations/identifier-urls.json. In SonarSource/sonar-analyzer-commons, the files list in sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json ships only README.md, secret-patterns.json and secret-exclusion-corpus.json. The 2.31.0-5284 tarball that SonarJS pins matches that: I downloaded it and it has no identifier-urls.json. So the module can't be resolved, and TypeScript build, bundling and the new test all fail. Commons has to generate and publish the inventory in a new npm release, and this repo has to move its pin to that release.

Here: Once Commons publishes a release that contains identifier-urls.json, bump @sonarsource/analyzer-commons-configurations in package.json and regenerate package-lock.json. Until then, keep this PR in draft so master doesn't break.

There: In sonar-analyzer-commons-configurations, generate identifier-urls.json (the domain-to-regex map) into the npm packaging dir, add it to files in src/main/packaging/npm/package.json, and publish a new version.

In SonarSource/sonar-analyzer-commons - sonar-analyzer-commons-configurations/src/main/packaging/npm/package.json:

"files": [
  "README.md",
  "secret-patterns.json",
  "secret-exclusion-corpus.json",
  "identifier-urls.json"
]

Was this helpful? React with 👍 / 👎


import identifierUrls from '@sonarsource/analyzer-commons-configurations/identifier-urls.json' with { type: 'json' };

// The inventory is maintained in Analyzer Commons; keep the matching semantics aligned.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Cross-PR: Padded URLs match in the Java filter but not in SonarJS · SonarSource/sonar-analyzer-commons #590

Breaks when sonar-analyzer-commons#590 is released with a populated inventory and the S5332 rule checks a URL string that has leading or trailing whitespace.

How

  1. CleartextProtocolFilter.isKnownIdentifier calls url.strip() before matching (commons/src/main/java/org/sonarsource/analyzer/commons/appsec/CleartextProtocolFilter.java:244).
  2. isKnownIdentifierUrl runs /^http:\/\/.../i directly on value with no trim, so any surrounding whitespace makes exec return null (packages/analysis/src/jsts/rules/S5332/identifier-urls.ts:34).
  3. So the same padded identifier URL is excluded by Java analyzers but still reported by SonarJS, even though the comment at identifier-urls.ts:20 says the matching semantics are kept aligned.

Fix

  • This PR: Trim value the same way as Java's strip() before running the authority regex, and add a test for a padded URL.
  • sonar-analyzer-commons#590: Nothing needed. Optionally list the whitespace stripping in the README's consumer matching rules.

Compared 4281d21 with sonar-analyzer-commons#590 at c0c18f9

Was this helpful? React with 👍 / 👎

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This PR is blocked due to unresolved code review findings.

Comment gitar unblock to override this block and allow merging.

Configure merge blocking · Maintainers can dismiss this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant